PMC

Boxplot of composition objective function results for different optimization algorithms.

Comparison between Reward-weighted Regression (RWR), sequential Reward-weighted Regression (sRWR) and Relevance Weighted Policy Optimization (RWPO). This time, the three algorithms optimize the distribution over trajectories with respect to all objectives. (A) Distribution after optimization with RWR, which uses a single reward function with a term for each objective. (B) Distribution after optimization using sRWR, which optimizes for each objective sequentially and has a reward function for each objective. (C) Distribution after optimization using RWPO, which uses the concept of relevance functions and optimizes for each objective sequentially with a reward function for each objective.

Pareto frontier for: (a) dual objective (UTS-E); (b) dual objective (UTS-H); and (c) dual objective (E-H) optimization.

Left panel: Standard mini-batch optimization uses a prescheduled learning rate, which determines the step size during optimization regardless of whether an optimization step leads to an improvement or not. Right panel: If line-search is enabled, the objective function is reevaluated on the same mini-batch and checked for improvement. If no improvement is achieved, the learning rate is reduced until either improvement is achieved or until the maximum number of line-search steps is reached.